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Abstract 

Educating farm labor supervisors about the regulations that govern agricultural operations and 
employment is critical to reducing unintentional violations of workplace safety and labor laws. 
Cooperative Extension can provide the training needed to professionalize this vital and diverse 
workforce. One challenge to providing adult education to a wide-ranging audience of labor 
contractors, crew leaders, and farm managers is implementing a program that meets varied 
needs and expectations and adapts to linguistic and cultural differences. This study combined 
quantitative and qualitative methods to measure learning and satisfaction in a bilingual 
Extension training program and to evaluate the relationship between educational outcomes and 
program components. The study found that attendees of both the English and Spanish trainings 
rated the quality of their experience as either high or very high regarding the intensity of 
learning, the likelihood of implementing what was learned, and their overall satisfaction with the 
program. Furthermore, post-test scores were significantly higher than pre-test scores for both 
groups. Nonetheless, English training attendees had significantly higher post-test scores than 
Spanish training attendees. The study concluded that successful agricultural educational 
program implementation is tied to adaptive teaching, staffing, and logistical strategies that 
respond to multicultural realities yet maintain pedagogical standards. 
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Enhancing the knowledge of farm labor supervisors concerning the rules and regulations 
that govern agricultural operations and employment is critical to reducing unintentional violations 
of work place safety and labor laws (Roka, Asuaje, & Thissen, 2011; Wiggins, 2009). A farm 
labor supervisor (FLS) may be a licensed labor contractor, crew leader, field foreman, farm 
manager or farm owner, but only labor contractors must obtain a valid certificate of registration 
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from federal and state agencies and, in most states, they are not required to receive additional 
instruction after their initial registration 7 (Roka et al., 2011; Roka, Olexa, Smallwood, Polopolus, 
& Fountain, 2009b). Nevertheless, supervisors in the field as well as growers and small 
subcontractors are all subject to the penalties of local, state, and federal laws protecting 
agricultural labor (see Roka et al., 2009a). Extension education programs can provide the 
training needed to professionalize this vital and diverse part of the workforce, integral to the 
nation’s multibillion dollar agricultural industry. Yet a challenge to providing agricultural 
education to FLSs, many of whom are foreign-born and have received different levels of 
education in the US and abroad, is implementing an efficacious training program that not only 
results in knowledge gains but meets varied needs and expectations and adapts to linguistic and 
cultural differences. 

Evaluation provides data that can ensure an agricultural education program offered 
through Cooperative Extension Services is targeted to client needs (Boone, Safrit, & Jones, 2002; 
McClure, Furhman, & Morgan, 2012), remains current (Yang, Fetsch, McBride, & Benavente, 
2009), and continues to attract audiences, particularly if it is to expand (Broun, Nilon, & Pierce II, 
2009). Nonetheless, the Extension literature reveals deficiencies in evaluation research (Duerden 
& Witt, 2012; Lamm & Israel, 2011; McClure et al., 2012) and has called for more rigorous 
analyses of program evaluation data (Braverman & Engle, 2009; Duerden & Witt, 2012; Lamm, 
Israel, & Harder, 2011). In response, this study combined ethnographic field methods and 
quantitative survey research to establish the effectiveness and quality of a bilingual Extension- 
based, adult-education program for FLSs, assessing the relationship between educational 
outcomes and program components. 

The program was developed in 2010 to provide a unified context under which 
agricultural labor contractors, crew leaders, field foremen, farm managers, and farm owners in 
Florida could better understand their roles and responsibilities in employing and managing 
farmworkers. With the guidance of an organizing committee made up of growers, labor 
contractors, worker advocacy representatives, and compliance officers from both federal and state 
agencies, faculty from a land-grant university designed a core curriculum comprised of four 
training modules in both English and Spanish: 1) Wage and Hour, 2) Human Resource (HR) 
Compliance, 3) Worker Protection Standard (WPS), Field Sanitation, and Food Safety, and 4) 
Safe Driving. The four modules are taught in a one-day certification Paining by a team of experts 
and trainers that includes specialized Extension agents, Department of Agriculture inspectors, 
Department of Transportation and Department of Labor officers, insurance adjusters, and former 
agricultural labor supervisors. The trainings have been offered since 2010 at county Extension 
offices and Research and Education Centers (RECs) throughout Florida. Yet a goal of the 
program is to expand its audience and educational modules in the future by offering the training 
in electronic format. 

The study reported here consisted of an external evaluation of the seven trainings offered 
during the fall of 2012. The evaluation was designed by an agricultural education program 
evaluation specialist and conducted by a bilingual (English/Spanish) team of agricultural 
education and communication faculty and research staff. 

Conceptual Framework 

Evaluation has evolved from being a necessity to being a priority in Extension, 
particularly as a tool for providing evidence of programmatic worth (Lamm & Israel, 2011). By 
documenting positive impacts on clients, Extension ensures its programs remain relevant to 


7 In California, farm labor contractors must annually enroll and participate in at least 8 hours of educational 
classes approved by the Labor Commissioner in partial fulfillment of state license renewal requirements 
(Cal. Lab. Code § 1684, 2011). 


Journal of Agricultural Education 


73 


Volume 55, Issue 3, 2014 



Morera et al 


Evaluating Extension-Based Adult Education... 


audiences and accountable to funders. Built into the design of an educational program, evaluation 
can also be used to examine the extent to which goals are being met, identify strengths and 
weaknesses, and provide information that can be used to improve outcomes (Frechtling, 2010; 
McNamara, 2006; Patton, 2008; Roucan-Kane, 2008). 

Duerden and Witt (2012) note that while outcome evaluations have become the norm 
across most Extension programs, they often fail to link observed outcomes to program 
components. This results in black-box outcome studies where program outputs are examined 
without examining internal operations and processes (Mowbray, Holter, Teague, & Bybee, 2003; 
Munoz, 2005; Patton, 2008). Patton (2008, p. 327) and Karachi, Abbott, Catalano, Haggerty, and 
Fleming (1999, p. 711) call for “getting inside and opening up the black box” by examining 
issues of program implementation. Incorporating program implementation findings can provide 
Extension educators with additional insight into how their programs are being conducted and help 
them identify effective practices (Duerden & Witt, 2012; Rossi, Lipsey, & Freeman, 2004). For 
example, a program’s targeted outcomes may be tied to the quality of its staff training. Thus, 
assessing implementation data can lead to a more accurate report and replicable methodology of a 
program’s production of observed outcomes. 

As early as 1978, Patton observed that outcome evaluations, represented by performance 
on standardized achievement tests, predominated in education while information about the actual 
nature of the programs being evaluated might be missing (Patton, 1978, p. 153). He emphasized, 
“where outcomes are evaluated without knowledge of implementation, the results seldom provide 
a direction for action because the decision maker lacks information about what produced the 
observed outcomes (or lack of outcomes)” (Patton, 1978, p. 155). An evaluation that focuses 
entirely on program outcomes, i.e. the extent to which the program was effective in changing 
behaviors or increasing knowledge, cannot answer why a program is effective or ineffective 
(Patton, 1978, p. 150). According to Patton, the ideal evaluation includes assessments of both 
implementation and outcome data. 

While Patton (1978, 2008) distinguishes between implementation and outcome 
evaluations, Frechtling (2010) further distinguishes between implementation and progress 
evaluations. Both are formative in that they provide information for program improvement, yet 
an implementation evaluation assesses whether a program is being conducted as planned whereas 
a progress evaluation assesses whether a program is meeting its ultimate goals (Frechtling, 2010). 
The latter “collects information to determine what the impact of the activities and strategies is on 
participants, curricula, or institutions at various stages of the intervention,” providing relatively 
early indicators of unexpected deviations that can be used to fine-tune it (Frechtling, 2010, p. 9). 

A set of questions guides each type of evaluation (Frechtling, 2010; Patton, 2008). 
Implementation evaluations are guided by questions that address, for example, educational 
program participant selection, recruitment activities, participation incentives, and program 
management. Progress evaluations, on the other hand, are guided by questions that address 
participant interest and performance, program expansion, and short-term participant impacts in 
light of long-term program goals. 

Additionally, Frierson, Hood, and Hughes (2010) maintain that taking into account the 
social and cultural milieu surrounding an educational program and its participants greatly 
enhances implementation and progress evaluation products. Culturally-responsive evaluation 
does not consist of a distinct set of steps, they argue, but rather represents a holistic framework 
that considers diverse voices and perspectives. The approach requires that evaluators “critically 
examine culturally relevant but often neglected variables in program design and evaluation” 
(Frierson et al., 2010, p. 76). Remaining responsive to the role of culture and the needs of the 
target population increases the effectiveness of a program and its evaluation. Moreover, Bryson 
and Patton (2010, p. 34) note that demonstrating cultural sensitivity and competence makes 
evaluation credible to people from different backgrounds. 
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Frierson et al.’s (2010) and Bryson and Patton’s (2010) arguments are relevant to 
university Extension programs which, generally, have been critiqued for their structural barriers 
to partnerships with underserved groups (e.g. Hassel, 2004; Klemme, Hausafus, & Shirer, 2005). 
Thering (2009) adds that in both program development and evaluation processes, Extension 
professionals must increasingly identify and overcome a spectrum of barriers formed by culture, 
class, ethnicity, race, and/or language if they are to produce intended educational and community 
capacity building outcomes. Thus, while analyzing the relationship between the components and 
outcomes of a program within a culturally-responsive framework increases the utility of 
evaluation products and, potentially, of the program itself—it remains a challenge to agricultural 
Extension educators. 


Purpose 

The puipose of this study was to evaluate the current effectiveness and quality of the 
bilingual Extension-based, adult-education program for FLSs (FLS training program) by 
measuring reception to, and knowledge gained in, the trainings offered during the fall of 2012 and 
by comparing educational outcomes to program components. The study assessed 
implementation, progress, and outcome data. In assessing the implementation of the program, the 
study was guided by the following questions: 

• Were the curriculum and training staff properly developed? 

• Were suitable recruitment strategies used? 

• Did the logistics of the Paining sessions meet and accommodate the practical needs of 
participants? 

• Were teaching materials and approaches tailored to participants? 

To appraise the progress of the program, the study addressed the following questions: 

• Are program participants enhancing their skills and gaining knowledge? 

• Are the trainings producing sufficient satisfaction to ensure participation is maintained or 
increases? 

To measure the educational outcomes of the program, the study focused on the following 
questions: 

• What is the level of participant proficiency in the training material? 

• What is the quality of participant experience regarding the intensity of learning, the 
likelihood of implementing what was learned, and their overall satisfaction with the 
program? 

These questions were addressed within a culturally-responsive framework that took into 
consideration cultural factors potentially impacting program components and educational 
outcomes. 


Methodology 


Data Collection 

Data were collected through participant observation and survey research (Bernard, 2006) 
during the seven trainings offered in both English and Spanish in the Florida cities of Homestead, 
Immokalee, Wimauma, Arcadia, Fort Pierce, Sebring, and Belle Glade from September 11 to 
October 31, 2012. Participant observation was used to address three of the eight guiding 
questions: whether the curriculum and training staff were properly developed, whether suitable 
recruitment strategies were used, and whether teaching materials and approaches were tailored to 
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participants. Survey research was used to gauge participant response to, and knowledge gained 
in, the trainings. The questionnaires addressed five of the eight guiding questions: whether the 
logistics of the training sessions accommodated the practical needs of participants, whether 
participants enhanced their skills and knowledge, whether the trainings were satisfying and well- 
attended, what the levels of participant proficiency in the training materials were, and what the 
quality of participant experience was regarding the intensity of learning, the likelihood of 
implementing what was learned, and their overall satisfaction with the program. 

Participant observation, conducted at all the trainings, was important to understanding the 
relationship between program implementation and educational outcomes. Because it involved not 
merely observing but taking part in the activities of the trainers and attendees, participant 
observation provided an insider’s view into the operation of, and reception to, the trainings. 
Members of the evaluation team sat in the audience through each of the trainings, observing and 
taking notes on training content, teaching methods and media, trainer expertise, and attendee 
response. They noted questions and comments brought up by members of the audience and how 
these were addressed by instructors. They interacted with program participants, trainers, 
sponsors, extension agents, and regulatory officers. They helped program staff set up training 
equipment and register attendees. Thus participant observation afforded data collection that 
complemented and contextualized survey research. Combining the two methods triangulated the 
data collected and increased the likelihood of producing trustworthy findings (Lincoln & Guba, 
1985). As Bernard (2006, p. 368) notes, “there are many things that people can’t or won’t tell 
you in answer to questions.” He emphasizes, “participant observation makes it possible to collect 
quantitative survey data...qualitative and quantitative data inform each other and produce insight 
and understanding in a way that cannot be duplicated by either approach alone” (Bernard, 2006, 
p. 356). 

Members of the evaluation team were bilingual and rotated as participant observers in the 
English and Spanish-led trainings. Rotation ensured the use of multiple observers across 
languages and improved the probability of credible results (Lincoln & Guba, 1985). At least two 
members attended each of the seven trainings, with one attending the sessions delivered in 
English and the other attending the sessions delivered in Spanish at each location. Over the 
course of the training season, each member of the evaluation team had the opportunity to observe 
the trainings delivered in both languages. 

Observations were recorded as descriptive field notes (Bernard, 2006) and shared with 
other members of the evaluation team via e-mail following each of the trainings. The notes 
included the participant observer’s account of training activities, materials, and methods as well 
as a list of questions and answers exchanged between the audience and instructors. The latter 
were written in the language and words in which the exchange occurred. Hence descriptive field 
notes documented details about teaching performance and audience response not easily captured 
by survey research. Later, these notes would help explain, for example, variations in survey 
responses and posttest scores. 

Survey questionnaires were developed in English and translated to Spanish by a bilingual 
native Spanish speaker with experience working with Florida crew leaders. The translations were 
reviewed by two biliterate members of the evaluation team. Surveys were piloted in Homestead, 
Florida to ensure comprehension by the target audience. 

Survey questionnaires were collectively administered by members of the evaluation team 
at the beginning and at the end of all trainings using Microsoft Office PowerPoint™ slides and 
TumingPoint™ software and equipment 8 . Gathered attendees were handed TurningPoint 
ResponseCard keypads (“clickers”), instructed in their use, and asked to respond to items 
appearing on the projector screen by pressing on their clickers the number corresponding to the 


8 TumingPoint software and equipment refers to audience response technology, i.e. ResponseCard keypads, 
receivers, and software, created by Turning Technologies to facilitate data collection and assessment. 
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best answer for each question. Items were formatted as multiple choice and true or false 
questions in the language the training was delivered in and read out loud from each of the 
PowerPoint slides. 

The pre-training questionnaire focused on demographic information and tested attendees 
on their familiarity with the training topics. It was composed of 19 items. The first 2 items, 
innocuous questions on gender and training location, were used as practice questions by 
displaying the results on the projector screen to ensure clickers were being used properly. 
Responses to the second item were also used to record attendance. The next 5 items included 
questions regarding years in agriculture, crops worked with, employment position, employees 
supervised, and attendance requirement. The remaining 12 items consisted of test questions 
regarding the topics of the four training modules: Wage and Hour; HR compliance; WPS, Field 
Sanitation and Food Safety; and Safe Driving. 

The post-training questionnaire was composed of 27 items and focused on self-reported 
knowledge levels, overall satisfaction with the training, and again tested attendees on their 
knowledge of the training topics. The first 8 items, constructed in pairs using Likert scales, 
consisted of before-and-after questions regarding knowledge gained in each of the core units of 
the training. The next 12 items consisted of test questions regarding the training topics. The last 
7 items, 5 of which were Likert items, included questions regarding satisfaction with the 
organization, expertise, and location of the training as well as questions regarding expectations, 
intensity of learning, likelihood of implementing what was learned, and recommending the 
training to others. 

Following each of the trainings, members of the evaluation team exchanged data and 
notes. Telephone meetings were held to discuss observations, trends, and logistical issues. 
Questionnaire items were modified to reflect realities on the ground as the study progressed. For 
example, some questionnaire items were shortened after the pilot training to facilitate response 
selection. 

Data Analysis 

Data collected through participant observation and recorded as descriptive field notes 
were logged by training location and module and organized according to evaluation research 
domains (Bernard, 2006). For instance, the quality and consistency of training content and 
delivery, nature of audience participation, and logistics were indexed for each of the four modules 
presented at each of the seven trainings. These indexes were used as references against which 
sessions could be compared and assessed and also provided a backdrop for survey results. As an 
example, the Wage and Hour and HR Compliance modules delivered by a local HR expert at the 
Immokalee Spanish training session generated a great deal of audience interest and interaction as 
did the WPS, Field Sanitation, and Food Safety module delivered by a regional specialized 
Extension agent. The quality of the training content, teaching methods, and audience 
participation indexed for those sessions later helped explain Immokalee’s relatively higher 
posttest scores compared to the other trainings delivered in Spanish (see Figure 4). 

Data collected through PowerPoint questionnaires were captured by TurningPoint 
receivers and automatically saved into a Microsoft Office Excel™ database. The TurningPoint 
software program coded data according to ResponseCard serial numbers and questionnaire item 
numbers and was also used to grade responses to the 12 pre-training and 12 post-training 
knowledge test questions. 9 Pre-training and post-training knowledge test scores, calculated by the 
program as a percentage of correct responses, were generated for ah attendees participating in the 
pre- and post-training surveys. 


9 Responses to one pre-training and one post-training knowledge test question, whose multiple-choice 
options were incomplete, were eliminated from final test scores. 
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Frequency distributions were used to capture trends in the responses to questions 
regarding demographics, satisfaction with the training, likelihood of implementing what was 
learned, and intensity of learning and to illustrate performance on post-training knowledge tests. 
Trends were disaggregated by attendance in trainings conducted in English and attendance in 
trainings conducted in Spanish. Available responses were counted for (n = 260) pre-training and 
(n = 218) post-training survey participants. 

In addition to summarizing the data through descriptive statistics, inferential statistics, in 
this case t-tests and multiple regression analysis, were used to draw conclusions about the values 
of the summary descriptions and, in turn, about the quality of the training program. Dependent 
and independent means t tests in SPSS™ were used to compare overall pre-training knowledge 
test scores to overall post-training knowledge test scores and to compare test scores for 
participants attending the trainings in English to test scores for participants attending the trainings 
in Spanish, respectively. This was done to assess statistical differences in the average test 
performance of participants before and after the trainings and between the English and Spanish 
sessions. A sample size of n = 157, the number of attendees who participated in both the pre- and 
post-training surveys and did not omit responses to demographic survey questions, was used for 
the t tests. 

Multiple regression analysis in SPSS was used to identify factors associated with post¬ 
training knowledge test scores. This was done to judge whether program or non-program 
variables contributed to differences in the average test performance of participants in dependable 
ways. This type of analysis was feasible because the dependent variable was a continuous, 
quantitative variable while the independent variables were either continuous quantitative 
variables or categorical variables dummy-coded as 1 or 0. The independent variables selected to 
estimate post-training knowledge test scores included pre-test scores, gender, crops worked with, 
years in agriculture, employment position, employees supervised, whether the training was taken 
in English, whether the training was taken as a requirement, and whether the training met 
attendee expectations. 

Variance inflation factors (VIF), used as a diagnostic tool, were calculated to detect the 
presence of multicollinearity among the predictor variables. Serious multicollinearity tends to 
make regression coefficients imprecise (Kutner, Nachtsheim, Neter, & Li, 2005). VIF values 
indicate “how ‘inflated’ the variance of the coefficient is, compared to what it would be if the 
variable were uncorrelated with any other variable in the [regression] model” (Allison, 1999, p. 
48). VIF values greater than 10 indicate severe multicollinearity (Kutner et al., 2005). Flowever, 
in this case, VIF values were all under five, with a mean of 2.33. Kutner et al. (2005) point out: 

The fact that some or ah predictor variables are correlated among themselves does not, in 
general, inhibit our ability to obtain a good [regression line] fit nor does it tend to affect 
inferences about mean responses or predictions of new observations, provided these 
inferences are made within the region of observations, (p. 283) 

Moreover, omitting from the regression model variables whose associations with the dependent 
variable are not significant can result in omitted variable bias, particularly when independent 
variables synergistically affect the dependent variable. 

Because not ah attendees remained for the duration of the trainings, only n = 195 
attendees participated in both the pre- and post-training surveys. Of those attendees, n = 38 
attendees did not provide responses regarding one or more of the regression model predictors. 
Thus, a sample size of n = 157 was used for the regression model. 
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Results 

Training Recruitment, Content, and Delivery 

Participant observation and discussions with staff revealed attendees were recruited 
through announcements posted on county Extension office and REC websites, in agricultural 
publications, and in pamphlets mailed to agribusinesses. Attendees were also recruited through 
Extension exhibits at agricultural events, such as the Citrus Expo, and through phone calls. 
Registration was available per training module, meaning attendees had the option to register for 
only those sessions they wished to participate in. The duration of each session was two hours and 
the price of each session was $10.00 on the registration forms. Attendance was expected to 
average 20 individuals per session. More than 300 individuals attended at least one class and 221 
individuals completed the day-long training program. The average number of attendees present at 
the beginning of the training sessions delivered in English and Spanish was 18 and 19, 
respectively. The average number of attendees present at the end of the training sessions 
delivered in English and Spanish was 12 and 19, respectively. 

Participant observation also revealed that training content and material, such as 
information presented on PowerPoint slides and handouts, were standardized in both languages 10 . 
Images were incorporated in the slides to facilitate comprehension and questions were 
incorporated to encourage discussion. Trainers were linguistically and culturally fluent in the 
communication and relational style of attendees. Several trainers were themselves foreign-born 
and had field and supervisory experience in agricultural work. The delivery of the presentations, 
however, varied by trainer. Some trainers did not use all of the information on the PowerPoint 
slides. Moreover, trainers in the sessions delivered in English often exhibited greater 
understanding of training topics and teaching methods than trainers in the sessions delivered in 
Spanish. 

Attendee Demographics 

Attendee survey responses to demographic questions (n = 260) indicated Paining 
attendance, as a whole, was mostly comprised of individuals who have worked in agriculture for 
more than 10 years. The sex ratio of Paining attendance was roughly 3 males to 1 female, 
overall. Almost half of all attendees were crew leaders and contractors. Twenty-two percent 
were employed as office staff, 10% as farm staff, 8% were employed in positions not listed on the 
survey, 7% were CEOs or owners, and 5% did not respond to the survey item regarding 
employment position. 

Many individuals who attended the trainings were directly responsible for a large number 
of employees. Twenty percent of attendees were directly responsible for more than 80 people. 
Fourteen percent were responsible for anywhere between 31 to 80 people. Another 14% were 
responsible for 21-30 people, 14% were responsible for 11-20 people, and 15% were responsible 
for up to 10 people. Fifteen percent were not responsible for any employees and 7% did not 
report how many employees they were responsible for. 

Crops attendees most worked with were fruit trees, vegetables, and melons. More than 
half of all attendees worked with fruit trees, such as citrus, mango, and avocado, and almost a 
third worked with vegetables and melons, such as peppers, tomatoes, zucchini, and cantaloupe. 
Four percent of attendees did not work directly with crops and 5% did not provide the 
information. The remaining attendees worked with berries, ornamentals, and other agricultural 
products such as corn, sugarcane, sorghum, and livestock. 


10 With the exception of several short videos shown in English during the Safe Driving sessions, the 
trainings were conducted in Spanish. 
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Attendee Reception to the Trainings 

Attendee survey responses rating their experience regarding the intensity of learning, the 
likelihood of implementing what was learned, and their overall satisfaction with the program (n = 
218) were summed and are illustrated in Figures 1 and 2. As a whole, attendees of both the 
English and Spanish training sessions rated the quality of their experiences as either high or very 
high. Discussions with attendees likewise revealed positive reception to the trainings. Attendees 
generally reported that the training material, both in English and Spanish, was useful. Participant 
observation indicated attendees showed interest in the topics during presentations with questions 
and discussions. 
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Figure 1. Relative Frequency Distribution of Attendees’ Quality of Experience Regarding the 
Intensity of Learning, the Likelihood of Implementing What Was Learned, and their Overall 
Satisfaction with the Trainings Delivered in English (n = 86). 
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Figure 2. Relative Frequency Distribution of Attendees’ Quality of Experience Regarding the 
Intensity of Learning, the Likelihood of Implementing What Was Learned, and their Overall 
Satisfaction with the Trainings Delivered in Spanish (n = 132). 


Attendee Knowledge Change at the Trainings 

The results of overall attendee performance on the post-training knowledge tests (n - 
218) are illustrated in Figures 3 and 4. The charts indicate the percent of correct responses and 
the percent of incorrect responses scored at the end of each training conducted in English and 
Spanish. For example, individuals who attended the Arcadia training conducted in English 
responded correctly to 80% of the questions. On the other hand, individuals who attended the 
Homestead and Sebring trainings conducted in Spanish responded correctly to 45% of the 
questions, respectively. Generally, individuals who attended the trainings conducted in English 
more frequently responded correctly to post-training test questions than did individuals who 
attended the trainings conducted in Spanish. The highest percent of correct responses at the 
trainings conducted in Spanish, 63% in Immokalee, was only 2 percentage points higher than the 
lowest percent of correct responses at the trainings conducted in English, 61% in Homestead. 
The average percent of correct responses scored at the trainings conducted in English and Spanish 
were 71 % and 54%, respectively. 
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Figure 3. Relative Frequency Distribution of Attendee Performance on (English) Post-Training 
Test (,n = 86). 
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Figure 4. Relative Frequency Distribution of Attendee Performance on (Spanish) Post-Training 
Test (n = 132). 
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Likewise, results of an independent means t test performed on (n = 157) post-training 
knowledge test scores (homogeneity of variance assumed, F (1, 155) = .19, ns) indicate that, on 
average, participants attending the trainings conducted in English had higher post-training 
knowledge test scores (M = 0.78 , SE = 0.02) than participants attending the trainings conducted in 
Spanish (M = 0.64, SE = 0.02). This difference was significant (t (155) = 5.74, p<.001; r = .42). 
Yet results of an independent means t test performed on (n = 157) pre-training knowledge test 
scores (homogeneity of variance assumed, F (1, 155) = .34, ns) indicate there was no significant 
difference (t (155) = 0.48, ns) between the scores of participants attending the trainings conducted 
in English (M = 0.58, SE = 0.02) and the scores of participants attending the trainings in Spanish 
(M = 0.57, SE = 0.02). Nonetheless, results of dependent means t tests indicate that, on average, 
post-training knowledge test scores were significantly higher than pre-training knowledge test 
scores for participants attending the trainings conducted in both English (post-test score M = 0.78, 
SE - 0.02; pre-test score M = 0.58, SE = 0.02; t (64) = -9.87, p<.001, r - .78) and Spanish (post¬ 
test score M = 0.64, SE = 0.02; pre-test score M = 0.57, SE - 0.02; t (91) = -4.62, p<.001, r = .44). 

Results of multiple regression analysis (n = 157) show whether program or non-program 
(i.e. demographic) factors were contributing to differences in English and Spanish post-training 
knowledge test scores. Results, illustrated in Table 1, indicate attendance in the trainings 
conducted in English was significantly and positively associated with post-training knowledge 
test scores and their best predictor (/;<().001). Pre-training knowledge test scores were also 
significantly and positively associated with the dependent variable (/?<().001). However, the 
remaining variables were not significantly associated with post-training knowledge test scores. 

Table 1 


Multiple Regression Analysis for Post-training Knowledge Test Score (n = 157) 


Predictor 

B 

SEB 

fi 

(Constant) 

0.44 

0.11 


Pretest Score 

0.34 

0.08 

.30* 

Attended training delivered in English 

0.12 

0.03 

.38* 

Male 

0.04 

0.03 

.12 

Work with fruit trees 

0.03 

0.03 

.09 

Work with vegetables and melons 

0.03 

0.04 

.10 

1 -5 years agricultural experience 

0.06 

0.06 

.12 

6-10 years agricultural experience 

0.06 

0.06 

.13 

More than 10 years agricultural experience 

0.03 

0.05 

.08 

Employed as a crew leader or contractor 

-0.04 

0.05 

-.13 

Employed as office staff 

0.05 

0.05 

.12 

Employed as farm staff 

-0.04 

0.05 

-.09 

Employed as a CEO or farmer 

-0.07 

0.07 

-.09 

Oversee over 80 workers 

0.03 

0.03 

.07 

Company required attendance 

-0.02 

0.03 

-.06 

Training met expectations 

-0.05 

0.08 

-.05 


Note: R 2 = .35. 

* p<.001. 
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Conclusions and Recommendations 

The results of this study point to several findings. First, and most importantly, individuals 
participating in the FLS training program are learning. Attendees scored significantly higher in 
post-training knowledge tests than in pre-training knowledge tests regardless of the language the 
training sessions were delivered in. 

Secondly, individuals are satisfied with the quality of their learning in the FLS training 
program. Attendees of both the English and Spanish training sessions rated the quality of their 
experience as either high or very high regarding the intensity of learning, the likelihood of 
implementing what was learned, and their overall satisfaction with the program. Components 
such as program organization, staff expertise, and training location contributed to this 
satisfaction. Moreover, the flexibility of the training schedule, the affordability of the sessions, 
and the affinity between the trainers and attendees were conducive to practical and motivational 
satisfaction. 

Nonetheless, a third finding is that attendees in the English training sessions had 
significantly higher post-test scores than attendees in the Spanish training sessions. The 
difference in knowledge gains can be partly explained by variations in the delivery of training 
material between languages. Trainers in the sessions delivered in Spanish, particularly those who 
did not help create the PowerPoint slides used in the program, were not consistent in presenting 
all of the information on the slides. Furthermore, the sessions delivered in English benefitted 
from a greater level of subject-matter expertise. 

Another potential explanation for the difference in knowledge gains is that demographic 
factors not captured by the study’s instruments may have partly contributed to learning 
differences; these factors may include diverse learning styles and needs, or just being more or less 
accustomed to traditional Extension learning environments. For example, the study’s surveys 
lacked items regarding education and income levels. However, there was no significant 
difference between pre-training knowledge test scores for attendees participating in the sessions 
delivered in English and attendees participating in the sessions delivered in Spanish. 

Thus, greater standardization of content delivery and staff development is recommended 
for the FLS training program. Staff should be further instructed in the teaching material, 
particularly regulations affecting agricultural personnel management and general administration. 
Staff should also be trained in effective pedagogical techniques. Yet the effectiveness of the FLS 
training program, underscored by its educational outcomes, is clearly tied to program components 
such as curricular quality, culturally and linguistically diverse staff, bilingual content 
development and delivery, and practical logistical strategies. 

Together, these results address the research questions that guided the evaluation. 
Participant observation revealed that despite delivery variations across languages, 1) the training 
modules were well developed, 2) recruitment strategies resulted in expected attendance rates, and 
3) teaching materials and media, because they were varied and facilitated comprehension through 
the use of images, were tailored for a variety of education levels. Survey responses indicated 4) 
the training sessions generally met and accommodated the practical needs of participants and 
resulted in overall high 5) satisfaction with the program 6) and quality of experience regarding 
the intensity of learning and the likelihood of implementing what was learned. Post-test scores 
revealed 7) participant levels of proficiency in the training material and indicated 8) the program 
results in significant knowledge gains. 
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Discussion and Implications 

FLSs are responsible for complying with complex labor laws and ensuring safe work 
environments for thousands of farmworkers. Providing them with adequate knowledge and skills 
is critical to upholding employment standards, preserving worker health, and reducing 
unintentional regulatory violations (Roka et al., 2011). By offering instruction on safety and 
compliance the FLS training program is helping to improve workplace conditions and 
professionalize the industry. 

Comprised of labor contractors, crew leaders, field foremen, farm managers and farm 
owners of various cultural backgrounds, FLSs represent a diverse group that is directly involved 
with managing vulnerable populations working in agriculture. This is significant not only for the 
agricultural industry but for Cooperative Extension. Hassel (2004), Klemme et al. (2005), and 
Thering (2009) have all noted that barriers formed by culture, class, ethnicity, race, and/or 
language differences impede partnerships with underserved groups. As this study has 
demonstrated, cultural and linguistic responsiveness is an essential program component that 
facilitates Extension collaborations with underserved audiences and ensures effective agricultural 
education. From an evaluative perspective, it is a replicable approach that can serve to improve 
the targeted outcomes of additional programs in other states. 

However, bilingual content and delivery do not guarantee expertise is translated. 
Linguistic and cultural responsiveness must be tied to the feedback mechanisms provided by 
program evaluation to ensure consistent curricular quality and equitable knowledge gains. This 
study contributes to both the evaluation and Extension literature by showing that successful 
agricultural educational program implementation is tied to adaptive teaching, staffing, and 
logistical strategies that respond to multicultural realities yet maintain pedagogical standards. 

In illustrating how the FLS training program’s educational outcomes are linked to its 
operation, the study confirms the necessity of conducting evaluations that analyze program 
implementation data (Duerden & Witt, 2003; Mowbray et al., 2003; Patton, 2008). Pretest- 
posttest design is important in gauging whether an agricultural education intervention is effective 
but not sufficient in assessing how or why. By using a mix of methods that combined participant 
observation with survey research, this study traced linkages between training delivery and 
participant impacts that may not have been captured by an outcome evaluation. 

The study also analyzed progress data by examining the FLS training program’s 
participant attendance rates, satisfaction levels, and knowledge gains. Frechtling (2010) has 
found progress data can provide timely identification of programmatic weaknesses. Here, the 
analysis provided an early indicator that participant knowledge gains could be improved in the 
trainings delivered in Spanish, a finding that can be used to fine tune the program’s achievement 
of meeting the educational needs of a critical segment of the industry’s labor force. Again, an 
outcome evaluation alone may not have provided the information necessary to strengthen the 
program nor addressed the critical question underlying any Extension effort: Is this program 
worthy of continuity or expansion? 

In this case, expanding the FLS training program should be a priority. Investigating the 
feasibility of providing the training in an electronic format for wider and more accessible 
statewide delivery is an area recommended for further study. Online video streaming and DVD 
recordings of the training may prove to be cost-effective methods of overcoming time restrictions 
potentially preventing increased program participation. 
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